A Review of Exposure Control Strategies for CAT and Potential Applications in MST

INTRODUCTION Computerized adaptive testing offers the advantages of more precise and efficient ability estimation and more flexible scheduling for testing when compared with conventional linear tests. Both item-level and testlet-level computerized adaptive tests have proved to be useful alternatives to the conventional paper-and-pencil tests. However, a potential problem is test security. In computerized adaptive testing, different examinees are presented with different sets of items and they are allowed to take the exam at different times. Consequently, some examinees may obtain knowledge of some test items prior to testing from those who have already taken the exam and seen the items. Thus, these examinees would be advantaged on the test and their test scores are no longer valid measures of their true trait levels. To maintain test security, attempts need to be made to avoid repeatedly administering the same items to examinees. Various exposure control strategies have been proposed to control overexposure of test items for item-level adaptive tests. Moreover, some of these strategies also aim at improving the usage of items with a tendency to serious underexposure. The purpose of this study is to provide a brief review of item exposure control strategies for item-level computerized adaptive tests (CATs). Furthermore, based on existing exposure control strategies, this study attempts to propose two designs that extend exiting exposure control strategies to multistage adaptive testing (MST). The MST model considered in this study is a three-stage computerized sequential adaptive testing (CAST) design, which is the administration model of the CPA exams. The rest of the paper is organized as follows: Section 1 provides a literature review of existing item exposure control strategies for CAT. Based on these strategies, section 2 proposes two designs that incorporate exposure control procedures for CAT into MST for the three-stage 3 CAST model of the CPA exams. A pilot study using a master pool of the CPA exams is presented. Finally, the concluding remarks are made in Section 3. Many high-stakes examinations are now being routinely administered in a computerized adaptive testing (CAT) (e.g., the Graduate Record Examination GRE, Armed Services Vocational Aptitude Battery) or multistage adaptive testing (MST) format (e.g., the Uniform Certified Public Accountants Examination, the Law School Admission Test). The main advantage of the CAT format is that it provides more efficient and precise estimation of the examinee's ability than does the conventional paper-and-pencil exam. In CAT, each successive item is selected so that …